The Hidden Information State Dialogue Manager: A Real-World POMDP-Based System

The Hidden Information State (HIS) Dialogue System is the first trainable and scalable implementation of a spoken dialog system based on the PartiallyObservable Markov-Decision-Process (POMDP) model of dialogue. The system responds to n-best output from the speech recogniser, maintains multiple concurrent dialogue state hypotheses, and provides a visual display showing how competing hypotheses are ranked. The demo is a prototype application for the Tourist Information Domain and achieved a task completion rate of over 90% in a recent user study. 1 Partially Observable Markov Decision Processes for Dialogue Systems Recent work on statistical models for spoken dialogue systems has argued that Partially Observable Markov Decision Processes (POMDPs) provide a principled mathematical framework for modeling the uncertainty inherent in human-machine dialogue (Williams, 2006; Young, 2006; Williams and Young, 2007). Briefly speaking, POMDPs extend the traditional fully-observable Markov Decision Process (MDP) framework by maintaining a belief state, ie. a probability distribution over dialogue states. This enables the dialogue manager to avoid and recover from recognition errors by sharing and shifting probability mass between multiple hypotheses of the current dialogue state. The framework also naturally incorporates n-best lists of multiple recognition hypotheses coming from the speech recogniser. Due to the vast number of possible dialogue states and policies, the use of POMDPs in practical dialogue systems is far from straightforward. The size of the belief state scales linearly with the number of dialogue states and belief state updates at every turn during a dialogue require all state probabilities to be recomputed. This is too computationally intensive to be practical with current technology. Worse than that, the complexity involved in policy optimisation grows exponentially with the number of states and system actions and neither exact nor approximate algorithms exist that provide a tractable solution for systems with thousands of states. 2 The Hidden Information State (HIS) Dialogue Manager The Hidden Information State (HIS) dialogue manager presented in this demonstration is the first trainable and scalable dialogue system based on the POMDP model. As described in (Young, 2006; Young et al., 2007) it partitions the state space using a tree-based representation of user goals so that only a small set of partition beliefs needs to be updated at every turn. In order to make policy optimisation tractable, a much reduced summary space is maintained in addition to the master state space. Policies are optimised in summary space and the selected summary actions are then mapped back to master space to form system actions. Apart from some very simple ontology definitions, the dialog manager has no application dependent heuristics. The system uses a grid-based discretisation of the